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ABSTRACT 



This paper describes a study that took place within the 
context of the Science: Parents, Activities, and Literature (Science PALs) 
project. Particularly, the study sought to determine answers to the following 
questions: (1) What are the internal consistencies and substantive, external, 

and structural validities of students' perceptions, teachers' self-reports, 
evaluations of videotaped classroom science teaching, and expert ratings? and 
(2) Can students' perceptions and attitudes and teachers' self-reports be 
used as acceptable surrogate measures for videotaped interactive 
constructivist science teaching? Findings suggest that instruments based on 
social constructivism do not completely and accurately document science 
teaching based on interactive-constructivist assumptions. (Contains 30 
references.) (WRM) 
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Introduction 

This study took place within the context of the Science: Parents, Activities, and Literature 
(Science PALs) Project. Science PALs was a four-year systemic reform effort collaboratively 
undertaken by the Science Education Center at the University of Iowa and the Iowa City 
Community School District and funded by the National Science Foundation (NSF) and the 
Howard Hughes Medical Foundation. The overarching goal of the project was to move teachers 
towards an interactive-constructivist model of teaching and learning that assumes a middle-of- 
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the-road interpretation of constructivism, where hands-on activities are used selectively and 
purposefully to challenge students’ ideas, promote deep processing, and achieve conceptual 
change. This model differs from the extreme interpretations of social constructivism and radical 
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constructivism. As many of the teachers in the project had little or no formal knowledge of 



interactive-constructivist principles, the project leaders sought to provide them with 

opportunities to examine these principles from the vantage points of learner and teacher. 

Problem 

A central problem with documenting any educational innovation is selecting or 
developing instruments that fairly represent the theoretical innovation with valid and reliable 
measures. The Science PALs’ version of interactive-constructivist science teaching and learning 
emphasized the shared roles of students, parents, and teachers regarding control, responsibilities, 
actions, and interactions. This perspective differs from traditional perspectives and other 
constructivist perspectives of science teaching and learning (Brooks & Brooks, 1993; Burry- 
Stock & Oxford, 1994; Yager, 1991). Traditional science instruction stresses absolute views of 
science, teacher control and structure, one-way communications and private mental activity, 
while many social constructivist perspectives stress post-modern views of science, group control 
and structure, two-way communications among the students directed at building consensus, and 
public mental activity. Therefore, it is necessary to use different documentation techniques and 
evaluation instruments (Searfoss & Enz, 1996). Common instruments may be used to document 
the basic features common among the perspectives, but additional instruments must be 
developed and used to document the unique features specific to any single perspective. 
Furthermore, it was decided that interactive-constructivist science instruction needed to be 
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documented from a variety of perspectives of the stakeholders (teachers, students, parents, etc.) 



utilizing multiple methods and data sources (demographics, instructional artifacts, teacher self- 
reports, classroom teaching, student perceptions, parental comments, etc.). Within this problem 
space, the following research questions were addressed: 

1. What are the internal consistencies and substantive, external, and structural validities of 
students’ perceptions, teachers’ self-reports, evaluations of videotaped classroom science 
teaching, and expert ratings? 

2. Can students’ perceptions and attitudes and teachers’ self-reports be used as acceptable 

surrogate measures for videotaped interactive-constructivist science teaching? 

Background 

Science teaching, science learning, and science teacher education research has enjoyed 
increasing popularity in recent years with the publication of the National Research Council’s 
National Science Education Standards (NRC, 1996), the National Board for Professional 
Teaching Standards (NBPTS, 1994), and the Report of the National Commission on Teaching 
and America’s Future (Darling-Hammond, 1996). These documents reaffirm the importance of 
teachers, teaching, and hands-on/minds-on learning as primary influences on students’ thinking, 
achievement, and science literacy. Furthermore, an analysis of the reform documents for 
language arts, mathematics, science, social studies, and technology revealed a common focus on 




“all” students, common learning outcomes of literacy and critical thinking, and common 
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instructional intentions regarding constructivism and authentic assessment (Ford, Yore, & 
Anthony, 1997). Unfortunately, little attention has been given to developing concise, clear 
definitions of these innovations and how these desired reforms will be documented. This section 
attempts to clarify what interactive-constructivist science teaching involves, how it can be 
documented using established and new techniques, and how these new instruments can be 
verified. 

Interactive-Constructivist Science Teaching in Elementary Schools 

Constructivism, a historical view of learning that embraces much of the contemporary 

cognitive, sociocultural and linguistic theories, has provided a powerful foundation for 
addressing people’s learning that behaviorism and cognitive development did not provide 
individually (Fosnot, 1996; Yager, 1991). Constructivism has encouraged educators to recognize 
the importance of ability, effort and prior performance, while also recognizing the potential 
influence of metacognitive awareness, self-regulation, misconceptions, sociocultural context, 
cultural beliefs, and interpretative frameworks. Unfortunately, the many interpretations of 
constructivism provide a “range of accounts of the processes by which knowledge construction 
takes place”, but few insights into how teachers can facilitate such learning with compatible 
teaching and assessment approaches (Driver, Asoko, Leach, Mortimer, & Scott, 1994, p. 5). The 
individual interpretations have some common assumptions and important differences: world 
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view, view of scientific knowledge, locus of mental activity, locus of structure/control, and 



discourse (Yore & Shymansky, 1997). 

The interactive-constructivist model of learning utilizes an ecology metaphor to illustrate 
learning in which dynamic interactions of prior knowledge, concurrent sensory experiences, 
belief systems, and other people in a sociocultural context lead to multiple interpretations that are 
verified against evidence and privately integrated (assimilated or accommodated) into the 
person’s knowledge network. Interactive-constructivist approaches differ from social 
constructivism (which utilizes a context metaphor to illustrate learning in which group dynamics 
lead to multiple interpretations that are verified by social negotiations resulting in consensus and 
common understanding at the group level) and radical constructivism (which utilizes an 
organism metaphor to illustrate learning in which intrapersonal deliberations and inner speech 
lead to equally valid unique interpretations that are internally assessed for personal consistency). 
Interactive-constructivist science teaching promotes a hybrid view of the world in which science 
is people’s attempt to search out, describe, and explain generalized patterns of events in the 
natural world and where these descriptions, explanations, and patterns are evaluated against 
evidence from nature. Constructing science understanding involves both public discussions to 
reveal and clarify alternatives and private reflections and reconciliation to integrate these new 
ideas into established knowledge networks. Interactive-constructivist science instruction utilizes 
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shared control between the teacher and students and two-way communication among students 



and teachers. The interactive-constructivist approach has the following attributes: 

( alignment among outcomes, instruction, resources, and assessment; 

( outcomes of conceptual change, conceptual growth, metacognitive strategic learning; 

( does not exclude just-in-time direct instruction embedded in a natural context of need; 

( supports big ideas/unifying concepts and habits of mind needed to attain scientific 
literacy; 

( requires students to gain ability to construct understanding, to think critically, to 
communicate their constructions, and to persuade others of their value or utility; 

( encompasses guided inquiry, learning cycles, conceptual change, and generative 
approaches; 

( and the teaching involves accessing, engaging, experiencing/exploring, 

justifying/rationalizing, consolidating/integrating old and new, and applying knowledge. 
If constructivism, like inquiry in the 1960s science education reform, is not clearly 
defined and anchored to classroom practices, it will fail to enhance science teaching and 
learning. Realizing the need for a well-defined instructional model, Henriques (1997) defined a 
prototypical interactive-constructivist elementary science teacher as having: 

a working knowledge about inquiry, the nature of science, and science topics in 
elementary school science. This content knowledge is married with age- 
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appropriate and topic-specific pedagogical knowledge (content-pedagogical 



knowledge) that informs instructional planning, classroom teaching, and 
assessment. The interactive-constructivist teacher is spontaneous, flexible, and 
anticipates learners’ interests, questions, and problems. This teacher is committed 
and reflective. The interactive-constructivist teacher teaches in a holistic, 
contextual manner with well-defined goals and cross-curricular connections. This 
teacher plans interactions with literature, activities, and prior experiences 
(includes misconceptions) in a supportive sociocultural context in which learners 
talk science, share alternative interpretations, and negotiate clarity. Children’s 
ideas are assessed, valued, and utilized to plan, to modify, and to challenge 
concrete experiences; and the resulting new ideas are consolidated and integrated 
with prior knowledge structures and related to their daily lives. The interactive- 
constructivist teacher implements a variety of strategies to meaningfully involve 
parents in their children’s science and in promoting science education. This 
teacher is a professional and leader responsible for professional development and 

an advocate for science in elementary schools. 

Documentation of Interactive-Constructivist Science Teaching in Elementary Schools 

Yager (1991) addressed the need for documenting constructivist science teaching by 

developing a self-check instrument consisting of 11 dipolar dimensions based on a science- 
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technology-society (STS) grid. He identified a variety of sociocultural groupings and problem- 



based tasks, their anticipated responses and results, and the associated teaching strategies as the 
basis for his constructivist learning model. Yager stated the “extent to which a teacher allows 
students to construct their own meaning will vary for teachers, individual students, and particular 
classrooms” (p. 56). Close inspection of the self-check instrument revealed that the dipoles 
represented a traditional perspective and a social constructivist perspective consistent with a STS 
orientation. Brooks and Brooks (1993) provided a list of eight pedagogical features dealing with 
curriculum, learning, teaching, assessment, and instructional groupings to contrast traditional 
classrooms and constructivist classrooms. Their interpretation of constructivism also appears to 
emphasize a social constructivist perspective. Burry-Stock and Oxford (1994) developed a 
science teaching evaluation model utilizing an expert-novice approach based on “a constructivist, 
student-centered perspective” (p. 278). Inspection of the dimensions and exemplars suggested 
that the constructivist perspective favors slightly a post-modern interpretation of science 
instruction. Collectively, the review of the related practice and literature identified four potential 
ways of documenting science instruction in elementary schools: expert ratings, classroom 
observations, students’ judgments, and teachers’ self-reports. 

Expert Ratings 

Supervision of teachers and evaluation of teaching effectiveness have historically relied 



on the judgments of legally recognized experts, such as superintendents, principals, directors of 



instruction, and content area coordinators. They are required to provide judgments of a teacher’s 
effectiveness based on their assessments of the teacher’s planning, administrative 
responsibilities, classroom management, teaching strategies, assessment techniques, and other 
identified features believed related to effective instruction. The experts’ judgments involve 
comparing their professional conceptions of teaching and their instructional expectations with 
actual classroom observations of the teacher’s teaching, professional interactions with the 
teacher, and artifacts of the teacher’s instruction. Occasionally, these judgments about science 
instruction were unreliable, and their validities were questioned because many of the legally 
identified experts lacked understanding of the desired teaching, the content area, the classroom 
context, and the associated types of evidence. 

Shymansky, Henriques, Chidsey, Dunkhase, Jorgensen, and Yore (1997) proposed the 
“professional development system” (PDS) to address these concerns about evaluating teaching 
effectiveness by identifying three important dimensions of instructional planning, classroom 
teaching, and leadership, and the associated points of evidence for each dimension. The PDS is 
based on the underlying assumptions of the interactive-constructivist perspective of science 
teaching, effective teaching (Dwyer, 1994; Shulman, 1986, 1990), and exemplary practices 
(Darling-Hammond, 1996). The PDS connects planning, science classroom practice, and 
leadership in elementary school to avoid the “tendency to ignore the substance of classroom life, 
the specific curriculum content and subject matter being studied” (Shulman, 1990, p. 53). Clearly 
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judgments about elementary school science teaching effectiveness must reflect the culture of 



elementary schools, the context of the elementary classroom, and the unique features of the 
scientific enterprise. 

In order to implement the PDS, definitions of quality within each dimension were 
developed in an iterative and collaborative manner — first relying largely on the literature. 
Second, conversations about proposed definitions of quality took place among the project staff, 
science advocates, and external consultants. The amended definitions resulting from these 
conversations were then re-evaluated against the research. Those definitions of quality that 
survived this process became the frame of reference for each performance standard. Based on 
experience and expertise in rating performance, confirmed by a growing literature in writing 
assessment in particular, the project staff elected to constrain each performance standard to four 
levels. The fourth, or highest level, is essentially the definition of quality for each dimension and 
collectively defines the desired prototype of an interactive-constructivist teacher (Henriques, 
1997). 

Classroom Observations 

Classroom observation systems like interaction analysis, science classroom assessment 
procedures, and macroanalysis techniques have been used to systematically observe and code 
classroom actions and communications. Each of these early systems was based on specific 



assumptions about teaching, such as verbal interactions, inquiry learning, or emerging strategies. 



Furthermore, the limitations of systems were transparent when used to document teaching that 



did not agree with the underlying assumptions of the instrument, such as using interaction 
analysis techniques to document hands-on inquiry science teaching. 

The Expert Science Teaching Educational Evaluation Model (ESTEEM) is based on 
student-centered learning and flexible classroom structures and is designed to assess expert 
science teaching within social constructivist classrooms (Burry-Stock & Oxford, 1994). The 
social constructivism perspective is somewhat implied by the fact that a high score is achieved 
when students interact with each other, discuss and test their own ideas, seek consensus, and 
share these ideas with the teacher and by its use to document STS teaching. Most examples of 
students’ activity is public with little evidence of the private reflection required by an interactive- 
constructivist approach. The constructs that each learner makes are influenced by interactions 
with others and cannot be separated from the sociocultural context. 

Burry-Stock’s (1995) Science Classroom Observation Rubric for ESTEEM has four 
major categories related to teaching: (a) facilitating the learning process from a constructivist 
perspective, (b) content-specific pedagogy (pedagogy related to students’ prior understanding 
and understanding of targeted concepts), (c) context-specific pedagogy (adjustments in strategies 
based on interactions with the students) and (d) content knowledge (knowledge of subject 
matter). Three to six dimensions inform each category. Each dimension has five levels of teacher 
performance ranging from novice (1) to expert (5) and are anchored in low inference 
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performance standards, which increases reliability and informs validity. The rubrics contain 
exemplars about students’ behaviors and actions in addition to the teacher’s behaviors and 
actions. The maximum score that can be earned by a teacher on the Classroom Observation 
Rubric is 90. 

The ESTEEM rubrics were reviewed by experts in the science education community. The 
external reviews supported the instrument’s face validity. The rubrics’ validity and reliability 
were explored using a group of expert teachers’ science classes. Forty-six Grade 4-8 teachers 
from seven states were selected to participate in the study. Burry-Stock and Oxford (1994) 
reported that the median for this group was 61, while the 25 th percentile was 50 and the 75 th 
percentile was 66. The means, standard deviations, and reliabilities were reported for the total 
instrument (57.30, 16.69, 0.91), facilitating learning processes (15.33, 4.81, 0.84), content- 
specific pedagogy (19.15, 5.12, 0.89), contextual knowledge (9.28, 2.70, 0.87), and content 
knowledge (13.54, 3.72, 0.80) categories. 

Analysis of the ESTEEM rubrics indicated that two categories (context-specific 
pedagogy and content knowledge) matched the interactive-constructivist model while the other 
two categories (facilitating the learning process from a constructivist perspective and content- 
specific pedagogy) differed slightly from the interactive-constructivist model. This alignment 
and the psychometrics for the ESTEEM rubrics justified its use in verifying the interactive- 



constructivist theory and related instruments. 



Students as Judges of Teaching Effectiveness 



The use of students’ perceptions of the constructivist teaching/leaming environment to 
measure effectiveness is not new. Fraser (1989) reviewed 60 studies of student perceptions of 
construe tivi si teaching environments. He argued that there were several advantages to using 
student perceptual measures rather than observational measures, including student perceptions 
are based on many lessons or classes, while peer/expert observations are based on limited 
numbers of observations; the information obtained is the pooled judgment of all the students as 
opposed to the single view of an observer; and the student perception is based on the teacher’s 
real behavior and therefore more important than inferred behavior based on observer judgment. 
Wilkinson (1989, p. 123) suggested that analysis of “student ratings of their teachers appeared to 
be as reliable as those undertaken by more experienced raters”. Wagenaar (1995, p. 68) argued 
that students “are best at detecting consumers’ perspectives on those teaching behaviors most 
noticeable to students”. 

Much of the recent work on student perceptions has been at the secondary school level 
with elementary schooling being overlooked (Goh & Fraser, 1995). Instruments developed at the 
secondary level, such as the Constructivist Learning Environment Survey (Chen, Taylor, & 
Aldridge, 1997), have used such factors as personal relevance, uncertainty, student negotiation, 
shared control, and critical voice to determine the level of student perception of the constructivist 



environment. Such factors are centered on the students’ beliefs that the teacher encourages them 



to negotiate meaning, they have some control of the learning, and the study of science is more 



than the authoritarian view put forward by the textbook (an absolutist view of science). Goh and 
Fraser’s (1995) study of elementary school science classrooms used the factors of leadership, 
helping/friendliness, understanding, student responsibility/freedom, uncertainty, dissatisfied, 
admonishing, and strictness as the bases for students’ perceptions of the learning environment. 
These factors focus on teacher behavior but not necessarily all appear to be reflective of a 
constructivist environment. When preservice elementary teachers were asked to judge the 
success of constructivist teaching approaches, they chose two primary factors: “students’ 
learning and the children’s attitudes toward science” (Stofflett & Stefanon, 1996, p. 15). This 
would indicate that instruments designed to measure elementary students’ perceptions of their 
teacher’s implementation of constructivist approaches should incorporate these factors. 

Self-Report of Teaching 

Self-report, self-check, and self-regulation are established goals of the reflective teacher. 
Self-evaluation — although a desired goal — has been questioned for its validity and reliability 
since it may emphasize intent rather than actual practice. Self-evaluation and self-report appear 
to provide better quality information when the instruments closely reflect the shared goals and 
understandings of the teachers reporting. Therefore, self-report instruments need to be custom 
designed to a specific task involving well-defined and commonly understood assumptions and 



should not be the single source of information for important decisions. 



Instrument Verification 



Instructional innovations require a close link between model verification and instrument 
verification. Therefore, constructivist teaching approaches must be assessed by instruments 



based on the same theoretical underpinnings, reflecting specific learning environments and 



disciplines, and not anchored to any single established reference (Geisinger, 1992; Royer, Cisero, 
& Carlo, 1993). Instrument validation (validity and reliability) is an accumulative inquiry 
process involving the theory, the prototype and the instrument (Anastasi, 1988; Geisinger, 1992; 
Messick, 1989). 

Validity can be considered in components: substantive, external, and structural (Yore, Craig, & 
Maguire, 1998). Substantive validity (face and construct) can be explored by objective expert 
analysis of the theory, prototype, and assessment instrument and by comparison of results of 
instruments to a commonly accepted reference. External validity (convergent and discriminate) 
can be examined by testing predictions (differences in groups expected to be different and detect 
changes known to exist) based on the underlying assumptions of the theory. Structural validity 
begins by assuming that reliable, valid data collected from the perspectives of the theory will 
exhibit the underlying assumptions of the theory. Factor analysis techniques can be used to 
examine the adequate fit of data to the fundamental structure of the model (Embretson, 1983). 
The goodness of fit between model and data can be explored by predetermining the number and 
unifying structure of the principal components revealed by the factor analysis (Loehler, 1987). 





Principal components of data that closely approximate the underlying assumptions of the model 



are taken as supportive evidence of the model. Reliability is an integral part of structural validity 

and intimately connected to factor analysis approach. 

Design 

The research questions were addressed using a case study of the science advocates for 
each of the 16 elementary schools in the Iowa City Community School District. These science 
advocates were elementary school teachers willing to serve as science leaders in their schools 
and to participate in the Science PALs project. Multiple methods were used to collect data on the 
science advocates’ implementation of the Science PALs’ goals (expert ratings), their classroom 
teaching (ESTEEM), their self-perceptions (self-reports), and their impact on students (students’ 
perceptions and attitudes). These data were collected during the 1995-96 school year. Complete 
data sets were developed for 14 science advocates since 2 science advocates left the school 
district and the project. 

Instruments 

Constructivist classrooms look different than their traditional counterparts. The students 
and teachers have different roles. As a result, traditional forms of teacher evaluation and 
measurement do not work well for constructivist classrooms (Searfoss & Enz, 1996). The first 
problem encountered in this study was to select and develop instruments that accurately reflected 
the interactive-constructivist theory. Techniques established to measure social constructivist 
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practices will not fully document the interactive-constructivist perspective. As a result, 
something else had to be used, while attempting to anchor the new instruments to the established 
instruments. The Professional Development System (PDS) was collaboratively developed by the 
science advocates, project staff, and external consultants (Shymansky, et al., 1997). The PDS 
guided the selection and development of instruments since it represented the underlying 
assumptions of the interactive-constructivist approach and was commonly understood by the 
science advocates, experts, and project staff involved in this study. 

Expert Ratings 

The advocates’ implementation of the Science PALS approach was globally assessed by 
four expert staff members who had been involved in the development and calibration of the PDS 
(Director of Professional Development, Science Coordinator, and two graduate student staff 
members). Each expert had strong science content background and considerable experience with 
elementary school teachers and science instruction. They were asked to rank order the 14 
advocates remaining in the project at the end of year 2 (1996) on their overall implementation of 
the interactive-constructivist approach, use of children’s ideas, use of children’s literature, and 
knowledge of reforms and misconceptions. The raters based their ratings on work with the 
science advocates in professional development activities and in their classrooms. There was 75% 
agreement (3 or more experts agreed within 1.5 positions in the rank order) among the four raters 
on the top 4 and bottom 4 advocates, while only 33% of the ratings of the middle 6 advocates 
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reached this level of agreement (Henriques, 1997). Therefore, an average rating was calculated 



for the four ratings. 



ESTEEM Ratings of Science Teaching 
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each science advocate. The videotapes were scored by two of three independent experts 
(Henriques, 1997). One of these experts had significant experience with the ESTEEM rubric 
from another research project. This expert trained the other two science educators to use the 
ESTEEM rubric. Drastic differences in scoring (±5 points on the total score) of a videotape 
resulted in discussion between the experts and rescoring when consensus could not be reached. 
The higher total ESTEEM score for the two science lessons was used to represent the advocate’s 
teaching for this analysis. The ESTEEM scores ranged from 39 to 85 with an average of 61.07. 
Students’ Perceptions of Science Teaching and Attitudes toward Science Learning 

Students’ perceptions of science teaching was composed of (a) view of constructivist 

approach, (b) parents’ interest, (c) teacher’s use of children's literature in science, and 
(d) relevance of science. Students' attitudes toward science learning was composed of (a) 
attitudes towards school science, (b) self confidence, (c) nature of science, and (d) science 
careers. These domains and subscales were assessed using Likert items to determine the 
students’ agreement, lack of awareness, or disagreement with specific statements about each 




factor. The items were developed by the project staff and external consultants to reflect the 



established features of the science reform and the project. The subscales were established using 



factor analyses techniques. Original items were scored as disagree (1), do not know (2), and 
agree (3) and were assigned to factors using a varimax approach with minimum loading weights 
of 0.30. Items not meeting this condition were deleted, resulting in a final Grades 1-2 survey of 
37 items, Grades 3-4 survey of 57 items, and Grades 5-6 survey of 72 items. Table 1 provides the 
number of items in each factor and the internal consistency based on data collected for Grades 1- 
2, 3-4, and 5-6 in the spring of 1996. Internal consistencies ranged from marginal (0.45-0.60) on 
18 data sets to reasonable (0.61-0.88) on 42 data sets. Generally, the instruments have reasonable 
validities (substantive and structural) and reliabilities for exploratory research, but further 

verification will result from this study. 

Teachers’ Self Reports 

The science advocates’ perceptions of their science planning, teaching, and assessment 
were assessed by Likert items designed to assess strong disagreement (1), disagree (2), neutrality 



Table 1 

Internal Consistencies of and Number of Items in the Likert Item Factors 
used to Assess Students’ Perceptions and Attitudes 



Scale and Factors 


Grade-Level Groupings 
1-2 3-4 5-6 


Perceotions of Science Teaching 


0.83(20) 


0.85(34) 


0.88(35) 


Constructivist Approach 


0.67(8) 


0.81(21) 


0.85(17) 


Parental Interest 


0.70(6) 


0.68(5) 


0.72(7) 


Use of Literature in Science 


0.52(3) 


0.49(3) 


0.61(5) 
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Relevance of Science 



Attitudes toward Science Learning 

Attitude toward School Science 

Self-concept 

Nature of Science 

Careers in Science 



0 . 50 ( 3 ) 

0 . 71 ( 17 ) 

0 . 58 ( 6 ) 

0 . 54 ( 3 ) 

0 . 60 ( 4 ) 

0 . 68 ( 4 ) 



0 . 56 ( 5 ) 

0 . 79 ( 23 ) 

0 . 74 ( 5 ) 

0 . 64 ( 6 ) 

0 . 53 ( 9 ) 

0 . 72 ( 3 ) 



0 . 74 ( 6 ) 

0 . 84 ( 37 ) 

0 . 81 ( 21 ) 

0 . 63 ( 6 ) 

0 . 51 ( 7 ) 

0 . 79 ( 4 ) 



(1996 data: N t = 2552, N,. 2 =831, N 3 . 4 = 722, N 5 . 6 = 999) 



(3), agree (4), or strong agreement (5) with specific ideas on a 5-point scale. The items were 
developed by the project staff and external consultants to reflect the underlying assumption of 
the project. A factor analysis of these items revealed 32 items with reasonable factor loadings 
(0.30) and conceptual unity fit a 3-factor solution. The first factor consisted of 17 items related to 
using children’s ideas in planning and teaching with an internal consistency of 0.88. The second 
factor consisted of 10 items related to the application of science to the children’s daily lives with 
an internal consistency of 0.82. The third factor consisted of 5 items related to the use of print 
resources with an internal consistency of 0.73. These three self-report dimensions closely parallel 
the pedagogical emphasis of Science PALs — using children’s ideas, relevance, and using 
children’s literature. The instrument was judged to have reasonable validities (substantive and 
structural) and reliabilities for exploratory research. 



Data Analyses and Results 



The research focus of this study was to verify the use of students’ perceptions and 
attitudes and teachers’ self-report information as measures of interactive-constructivist science 
teaching in elementary schools. The analyses provide descriptive data, correlations, ANOVAs, 
and T-tests for 14 science advocates who have been involved in the Science PALs project. The 
correlations between these measures and established instruments (expert ratings and ESTEEM 
ratings) provide an indication of substantive validity (construct). Differences in ESTEEM scores, 
perceptions, attitudes, and self-report information for 3 groups of advocates based on the expert 
ratings were tested using Analyses of Variance (ANOVA) and T-tests as indications of external 
validity. Tables 2, 3, and 4 provide descriptive statistics, correlations, summary ANOVAs, and 
pair-wise T-tests for each measure. 

The expert ratings were based on the judgments of the 4 well-informed science educators 
and on numerous encounters with the 14 science advocates’ instructional planning, teaching, and 
leadership. There was 68% agreement (±1.5 positions in rank order for 3 of the 4 experts) among 
the science educators. The variability was expected since each expert viewed the science 
advocates’ implementation from slightly different perspectives, but their collective judgment 
(average rating) was believed to be the best indicator of the science advocates’ overall 
implementation of the interactive-constructivist approach. 




The ESTEEM scores indicate that the science advocates do not fully reflect the social 



constructivist perspective implicit in the instrument. The average ESTEEM score for the 
advocates teaching (61.07) is slightly higher than the average ESTEEM score for the 46 expert 
science teachers (57.30) used to norm the instrument (Burry-Stock & Oxford, 1994). Only one 
advocate with a score of 85 appears to approximate the teaching strategies implied by the 
ESTEEM rubric. Most science advocates (12) have middle-of-the-road scores (55 to 75) that are 
characteristic of a shared-control but learner-focused teaching style implied by the prototypical 
interactive-constructivist teacher. The interactive-constructivist teaching approach has many 
common expectations with the ESTEEM model, but it is unlikely that an interactive- 
constructivist teacher would score in the 80-90 range on the ESTEEM rubric. 

The students’ perceptions of their teacher’s teaching were slightly positive to positive 
(2.08-2.67), while students’ attitudes toward science learning were somewhat more positive 
(2.19-2.63). Teachers self-reported use of students’ ideas, applications of science to the 
children’s world, and use of print resources were positive (3.63-4.07). Close inspection of 
Table 2 illustrates a general pattern among the science advocates identified by the experts’ 
ratings as low-level implementors, middle-level implementors, and top-level implementors in 

Table 2 

Means of Science PALs Advocates for Specific Measures of Constructivist Teaching 

Level of Implementation 

Measure Low (N = 4) Middle (N = 6) Top (N = 4) 

OO 

fc <J> 



BEST COPY AVAILABLE 



ESTEEM 


55.00 


61.83 


71.00 


Students’ Perception of Science Teaching (SPST) 


2.40 


2.33 


2.50 


Students’ View of Constructivism (SVC) 


2.66 


2.58 


2.77 


Parental Interest (SPI) 


1.99 


1.93 


2.14 


Use of Literature (SUL) 


1.55 


1.50 


2.20 


Relevance (SR) 


2.71 


2.68 


2.41 


Students’ Attitude toward Science Learning (SASL) 


2.46 


2.41 


2.42 


Students’ Attitude toward School Science (SASS) 


2.20 


2.19 


2.64 


Students’ Self-concept (SSC) 


2.66 


2.67 


2.53 


Nature of Science (SNS) 


2.84 


2.73 


2.06 


Careers in Science (SCS) 


2.08 


2.08 


2.47 


Teachers using Children’s Ideas (TUCI) 


3.49 


3.43 


3.66 


Teachers’ Applications to Children’s World (TACW) 


3.81 


3.83 


3.73 


Teachers using Print Resources (TUPR) 


3.25 


3.21 


4.04 



which the middle-level group frequently has lower student perceptions, student attitudes, and 
self-reports than do the low-level and top-level implementors. The top-level implementors have 
the highest values on 8 measures while the low-level implementors have the highest value on 5 
measures. 
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Table 3 

Inter-measure Correlation Matrix 
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denotes p < 0.05 
denotes p < 0.01 
denotes p < 0.001 
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Table 4 

Summary ANOVAs and Pair-wise T-tests for Three Groups of Science Advocates 
Based on their Perceived Implementation of Science PALs 
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The correlation between the ESTEEM scores and the expert ratings was significant 



(p < 0.01). The inter-measure correlations revealed disappointing relationships between 
ESTEEM and students’ perceptions, students’ attitudes, and teacher’s self-report information. 
The relationships between expert ratings and the students’ perceptions, students’ attitudes, and 
teacher’s self-report information is somewhat more promising. These results support the 
expectation that the ESTEEM instrument emphasized a slightly different perspective of 
constructivism than the interactive-constructivist approach. Furthermore, these results suggest 
that the students’ perceptions and attitudes and the teacher’s self-report instruments need fine- 
tuning to increase their substantive validities. 

Significant (p < 0.05) main effects in the ANOVAs were found for ESTEEM, students’ 

view of constructivist approach, students’ perception of use of literature in science, students’ 
attitude toward school science, students’ attitude toward the nature of science, students’ attitude 
toward careers in science, and teachers’ report of using print resources. The pair-wise t-test 
comparisons for these significant main effects reveal that 7 differences between the top-level 
implementors and middle-level implementors account for most of the main effects. Most 
frequently, the difference favored the top-level implementors (except students’ self-concept in 
science). Six differences between the top-level and low-level implementors were found 
significant. All but one difference (students’ self-concept in science) favored the top-level 
implementors. The only significant difference between middle-level and low-level implementors 
was for the total ESTEEM scores. 




Discussion 



This study supported the anticipation that instruments based on social constructivism 
would not completely and accurately document science teaching based on interactive- 
constructivist assumptions. The correlations between experts’ ratings and ESTEEM (r = 0.68) 
and the ANOVA and pair-wise t-tests of ESTEEM differences for groups specified by their 
expert ratings were significant, but sizable amounts of variance (54%) were not accounted for by 
the social constructivist-based instrument. 

This study also revealed that some students’ perceptions of science teaching (view of 
constructivism, parental interest, and using literature in science) and attitudes toward science 
learning (attitudes toward school science and careers in science) have reasonable validities and 
reliabilities to use as indicators of interactive-constructivist teaching. Teachers’ self-reports of 
using children’s ideas and use of print resources also have potential as indicators of interactive- 
constructivist teaching. Students’ perceptions of relevance and self-concept, self-concept in 
science, and attitudes toward nature of science and teachers’ self-report of applications of science 
to the children’s daily lives appear to distract from the instruments’ substantive and external 
validities. Eliminating or revising these dimensions should improve the overall utility of these 
instruments. 

Analysis of each dimension (item) within the 4 categories of the ESTEEM rubric 
suggests that some items could be revised or the exemplars revised to more closely reflect an 




interactive-constructivist perspective of elementary school science teaching. Revisions would 



require new expert exemplars for student engagement in activities, novelty, textbook 
dependency, student relevance, and higher order thinking skills. Another solution might be to 
establish an ideal ESTEEM score for the prototypical interactive-constructivist teacher (i.e., 85) 
and to express ESTEEM scores as a positive or negative deviation score about the idealized 
ESTEEM score. This would allow changes in teaching to be defined as growth toward the ideal 
score. 

This study of 14 science advocates’ teaching, their students’ perceptions and attitudes, and their 
self-reports indicates that students can detect some differences and some self-reported 
information matches external judgments. If the two instruments are revised, they can be efficient 
and effective surrogate measures of interactive-constructivist science teaching in elementary 
schools. 
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